Analyses of model fit and robustness. A new look at the PISA scaling model underlying ranking of countries according to reading literacy.
نویسندگان
چکیده
This paper addresses methodological issues that concern the scaling model used in the international comparison of student attainment in the Programme for International Student Attainment (PISA), specifically with reference to whether PISA's ranking of countries is confounded by model misfit and differential item functioning (DIF). To determine this, we reanalyzed the publicly accessible data on reading skills from the 2006 PISA survey. We also examined whether the ranking of countries is robust in relation to the errors of the scaling model. This was done by studying invariance across subscales, and by comparing ranks based on the scaling model and ranks based on models where some of the flaws of PISA's scaling model are taken into account. Our analyses provide strong evidence of misfit of the PISA scaling model and very strong evidence of DIF. These findings do not support the claims that the country rankings reported by PISA are robust.
منابع مشابه
Implicational Scaling of Reading Comprehension Construct: Is it Deterministic or Probabilistic?
In English as a Second Language Teaching and Testing situations, it is common to infer about learners’ reading ability based on his or her total score on a reading test. This assumes the unidimensional and reproducible nature of reading items. However, few researches have been conducted to probe the issue through psychometric analyses. In the present study, the IELTS exemplar module C (1994) wa...
متن کاملDesigning and Developing a Test for Cognitive Competencies of the Iranian Students’ Mathematics Literacy based on PISA Studies
Since the establishment of formal education in Iran, there has always been an emphasis on the application of mathematics in real life situation. To measure students’s competencies in applying mathematics in real life situations, there is a need to design a test with this purpose. During the current decade, PISA has been conducted in various countries to measure sudents’ competencies needed for ...
متن کاملEstimating IDF based on daily precipitation using temporal scale model
The intensity –duration –frequency (IDF) curves play most important role in watershed management, flood control and hydraulic design of structures. Conventional method for calculating the IDF curves needs hourly rainfall data in different durations which is not extensively available in many regions. Instead 24-hour precipitation statistics were measured in most rain-gauge stations. In this stud...
متن کاملPISA test format assessment and the local independence assumption
Large-scale assessments of reading comprehension, notably OECD’s Programme for International Student Achievement (PISA) and IEA’s Progress in Reading Literacy Study (PIRLS), generally use paper-and-pencil tests in which a reading passage, with different questions based on it, is presented to the student. The PISA mathematics and science literacy tests also consist of a hierarchically embedded s...
متن کاملMathematical and Scientific Literacy Around The World
PISA, the OECD’s international program of assessment of reading, scientific and mathematical literacy, aims to assess the knowledge and skills that students have acquired at school and their ability to use them in everyday tasks and challenges. It also uses questionnaires to gather data on students’ attitudes to learning and the conditions of schooling. Since 2000, PISA has tested the scientifi...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- Psychometrika
دوره 79 2 شماره
صفحات -
تاریخ انتشار 2014